As an increasing number of applications and services are being made available over networks such as the Internet, an increasing number of content, application, and/or service providers are turning to technologies such as cloud computing. Cloud computing, in general, is an approach to providing access to electronic resources through services, such as Web services, where the hardware and/or software used to support those services is dynamically scalable to meet the needs of the services at any given time. A user or customer can obtain access to various services through the cloud, or across at least one network, and thus does not have to purchase and maintain the hardware and/or software associated with the services. In some cases, a computing device for a user might include several different applications that each requires access to one or more of these services. Each time one of these applications wants to access one of the services, a connection is established to allow for the necessary communications.
Establishing and tearing down a large number of connections, however, leads to various inefficiencies. Each time a single request and single response connection is opened and/or closed there is some computational overhead involved. Further, there can be handshakes with the destination service on various network layers that each add a delay before a connection is opened. In some systems ongoing connections can be used to a destination service, but such connections introduce additional overhead because the connection must be kept alive with heartbeats, and unless the number of communications justifies the overhead a significant amount of time and resources can be wasted.